Tuple Packing: Efficient Batching of Small Graphs in Graph Neural Networks

When processing a batch of graphs in machine learning models such as Graph Neural Networks (GNN), it is common to combine several small graphs into one overall graph to accelerate processing and remove or reduce the overhead of padding. This is for example supported in the PyG library. However, the sizes of small graphs can vary substantially with respect to the number of nodes and edges, and hence the size of the combined graph can still vary considerably, especially for small batch sizes. Therefore, the costs of excessive padding and wasted compute are still incurred when working with static shapes, which are preferred for maximum acceleration. This paper proposes a new hardware agnostic approach -- tuple packing -- for generating batches that cause minimal overhead. The algorithm extends recently introduced sequence packing approaches to work on the 2D tuples of (|nodes|, |edges|). A monotone heuristic is applied to the 2D histogram of tuple values to define a priority for packing histogram bins together with the objective to reach a limit on the number of nodes as well as the number of edges. Experiments verify the effectiveness of the algorithm on multiple datasets

Extreme Acceleration of Graph Neural Network-based Prediction Models for Quantum Chemistry

Molecular property calculations are the bedrock of chemical physics. High-fidelity \textit{ab initio} modeling techniques for computing the molecular properties can be prohibitively expensive, and motivate the development of machine-learning models that make the same predictions more efficiently. Training graph neural networks over large molecular databases introduces unique computational challenges such as the need to process millions of small graphs with variable size and support communication patterns that are distinct from learning over large graphs such as social networks. This paper demonstrates a novel hardware-software co-design approach to scale up the training of graph neural networks for molecular property prediction. We introduce an algorithm to coalesce the batches of molecular graphs into fixed size packs to eliminate redundant computation and memory associated with alternative padding techniques and improve throughput via minimizing communication. We demonstrate the effectiveness of our co-design approach by providing an implementation of a well-established molecular property prediction model on the Graphcore Intelligence Processing Units (IPU). We evaluate the training performance on multiple molecular graph databases with varying degrees of graph counts, sizes and sparsity. We demonstrate that such a co-design approach can reduce the training time of such molecular property prediction models from days to less than two hours, opening new possibilities for AI-driven scientific discovery

Generating QM1B with PySCFIPU_{\text{IPU}}

The emergence of foundation models in Computer Vision and Natural Language Processing have resulted in immense progress on downstream tasks. This progress was enabled by datasets with billions of training examples. Similar benefits are yet to be unlocked for quantum chemistry, where the potential of deep learning is constrained by comparatively small datasets with 100k to 20M training examples. These datasets are limited in size because the labels are computed using the accurate (but computationally demanding) predictions of Density Functional Theory (DFT). Notably, prior DFT datasets were created using CPU supercomputers without leveraging hardware acceleration. In this paper, we take a first step towards utilising hardware accelerators by introducing the data generator PySCF$_{\text{IPU}}$ using Intelligence Processing Units (IPUs). This allowed us to create the dataset QM1B with one billion training examples containing 9-11 heavy atoms. We demonstrate that a simple baseline neural network (SchNet 9M) improves its performance by simply increasing the amount of training data without additional inductive biases. To encourage future researchers to use QM1B responsibly, we highlight several limitations of QM1B and emphasise the low-resolution of our DFT options, which also serves as motivation for even larger, more accurate datasets. Code and dataset are available on Github: http://github.com/graphcore-research/pyscf-ipuComment: 15 pages, 7 figures. NeurIPS 2023 Track Datasets and Benchmark

Reducing Down(stream)time: Pretraining Molecular GNNs using Heterogeneous AI Accelerators

The demonstrated success of transfer learning has popularized approaches that involve pretraining models from massive data sources and subsequent finetuning towards a specific task. While such approaches have become the norm in fields such as natural language processing, implementation and evaluation of transfer learning approaches for chemistry are in the early stages. In this work, we demonstrate finetuning for downstream tasks on a graph neural network (GNN) trained over a molecular database containing 2.7 million water clusters. The use of Graphcore IPUs as an AI accelerator for training molecular GNNs reduces training time from a reported 2.7 days on 0.5M clusters to 1.2 hours on 2.7M clusters. Finetuning the pretrained model for downstream tasks of molecular dynamics and transfer to a different potential energy surface took only 8.3 hours and 28 minutes, respectively, on a single GPU.Comment: Machine Learning and the Physical Sciences Workshop at the 36th conference on Neural Information Processing Systems (NeurIPS

An Empirical Study of HR Practices and Employee’s Engagement in Banking Sector

Companies around the world are grappling about how to improve employee engagement in an organization. For businesses to achieve long-term sustainability and well-being, employees must be engaged. Employees accept the job that could help them go through the ladder of their needs starting from the basic needs like security up to self-enhancement, for instance, self-ego or self-worth. Social exchange theory has been selected for this study, because this theory provide knowledge and understanding which is related to the study of employee engagement. This study used quantitative approach. The samples of this research were 132 employees of private banks in Sana’a, Yemen. Data retrieval used a questionnaire and the responses are accumulated through the structured questionnaires. Keywords: employee engagement, reward & recognition, job securit

Principles of Modern Chemistry

xxxvii, 1132 p. : Ill.; 28 cm

Indapamide Increases IRS1 Expression and Modifies Adiponectin/NLRP3/PPARγ Crosstalk in Type 2 Diabetic Rats

The current study aimed to evaluate the anti-diabetic effects of canagliflozin (CANA) and indapamide (INDA) and their impacts as adiponectin modulators in experimentally induced type 2 diabetes mellitus (T2DM). T2DM was associated with a significant rise in blood glucose level and HbA1C%, andreduced adiponectin and insulin secretions. Moreover, the malondialdehyde (MDA) contents in both the epididymal adipocytes and soleus muscle significantly escalated, while the total antioxidant capacity (TAC) and epididymal adipocyte Nrf2 expression significantly declined. Moreover, serum TNF-α, epididymal adipocyte’s NOD-like receptor protein 3, NLRP3, NF-κB and CD68 expressions markedly escalated, and serum IL-10 significantly declined. Furthermore, there was a significant escalation in PPARγ expression in epididymal adipocytes, with a significant reduction in soleus muscle’s expression of IRS1. CANA and INDA treatments markedly reduced blood glucose levels, increased adiponectin and insulin secretion, enhanced anti-oxidant defenses, and reduced oxidative burden, with marked anti-inflammatory impact. Interestingly, the impact of indapamide on DM indices and oxidative and inflammatory changes was comparable to that of canagliflozin. Nevertheless, indapamide had a superior effect compared to canagliflozin on HbA1c%, expression of IRS1 and reduction of NF-κB and CD68 expressions. INDA could be effective in regulating T2DM, with underlined anti-diabetic, antioxidant, and anti-inflammatory properties. INDA increased IRS1 expression and modified adiponectin/NLRP3/PPARγ crosstalk. The impacts of INDA are comparable to those of the standard anti-diabetic drug CANA

Comparative study of ionic bombardment and heat treatment on the electrical behavior of Au/GaN/n-GaAs Schottky diodes

International audienc

A new model of thermionic emission mechanism for non-ideal Schottky contacts and a method of extracting electrical parameters

International audienc